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Abstract 

Forecasts of influenza activity in human populations could help guide key preparedness tasks. We conducted a scoping 
review to characterize these methodological approaches and identify research gaps. Adapting the PRISMA methodology for 
systematic reviews, we searched PubMed, CINAHL, Project Euclid, and Cochrane Database of Systematic Reviews for 
publications in English since January 1, 2000 using the terms "influenza AND (forecast* OR predict*)", excluding studies that 
did not validate forecasts against independent data or incorporate influenza-related surveillance data from the season or 
pandemic for which the forecasts were applied. We included 35 publications describing population-based (N = 27), medical 
facility-based (N = 4), and regional or global pandemic spread (N = 4) forecasts. They included areas of North America 
(N = 15), Europe (N = 14), and/or Asia-Pacific region (N=4), or had global scope (N = 3). Forecasting models were statistical 
(N = 1 8) or epidemiological (N = 1 7). Five studies used data assimilation methods to update forecasts with new surveillance 
data. Models used virological (N = 14), syndromic (N = 13), meteorological (N = 6), internet search query (N =4), and/or other 
surveillance data as inputs. Forecasting outcomes and validation metrics varied widely. Two studies compared distinct 
modeling approaches using common data, 2 assessed model calibration, and 1 systematically incorporated expert input. Of 
the 17 studies using epidemiological models, 8 included sensitivity analysis. This review suggests need for use of good 
practices in influenza forecasting (e.g., sensitivity analysis); direct comparisons of diverse approaches; assessment of model 
calibration; integration of subjective expert input; operational research in pilot, real-world applications; and improved 
mutual understanding among modelers and public health officials. 
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Introduction 

Seasonal influenza epidemics caused by influenza A and B 
viruses occur annually during the winter in temperate regions, 
resulting in around 3-5 million cases of severe illness and 250,000- 
500,000 deaths worldwide each year [1]. In contrast to seasonal 
influenza, novel influenza A strains capable of sustained person-to- 
person transmission arise occasionally. These novel strains may 
evade existing antibody immunity and give rise to pandemic 
outbreaks. For example, the 1918 pandemic caused around 20-40 
million deaths [2], while pandemics in 1957 and 1968 involved 
many infections but fewer deaths than in the 1918 pandemic. A 
2009 pandemic strain, influenza A(HlNl)pdm09, continues to 
circulate as a seasonal virus. 

Accurate forecasts of influenza activity based on predictive models 
could facilitate key preparedness actions, such as public health 
surveillance, development and use of medical countermeasures (e.g., 
vaccine and antiviral drugs), communication strategies, deployment 
of Strategic National Stockpile assets in anticipation of surge 
demands (e.g., ventilators), and hospital resource management 



(e.g., for staf and beds). Early in a potential pandemic, forecasts of 
international spread could help guide public health actions globally. 

Previous reviews have assessed influenza modeling (e.g., [3-6]), 
but to our knowledge only one focused specifically on the use of 
models to forecast influenza activity, as opposed to other 
important applications of influenza modeling (such as improving 
understanding of the epidemiological dynamics or evaluating 
control strategies). This recent review, by Nsoesie et al. [7], 
identified 16 studies that aimed to forecast influenza outbreaks at 
local, regional, national, or global level. To more systematically 
characterize influenza forecasting methods and applications, and 
identify promising approaches and research gaps, we conducted a 
scoping review of the peer-reviewed influenza forecasting litera- 
ture. We assess differences in methodological approaches and 
provide recommendations for future influenza forecasting models. 

Materials and Methods 

We adapted the PRISMA methodology [8] for our scoping 
review. In contrast to a systematic review, which focuses on a well- 
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defined research question and may include a narrow range of 
study designs, a scoping review addresses broader topics and may 
include various study designs [9]. We included studies that 
described methods to forecast future influenza activity in human 
populations using dynamic influenza-related surveillance data, and 
that tested the forecasting approach against independent data (real 
or simulated). We defined "dynamic" data as data collected during 
an epidemic or pandemic to make predictions about its subsequent 
course. We excluded studies that predicted current influenza 
activity not observed at the time of prediction because of reporting 
delays (sometimes called "nowcasting"). 

We searched PubMed, CINAHL, Project Euclid, and Cochrane 
Database of Systematic Reviews for publications in English since 
January 1, 2000 using the terms "influenza AND (forecast* OR 
predict*)" in any field, and analyzed abstracts of returned 
publications to identify candidates for full-text review (i.e., studies 
for which inclusion criteria were met or for which it was not 
possible to determine whether inclusion criteria were met). We 
also manually searched the reference lists of included papers, our 
bibliographies, and the International Journal of Forecasting, and 
considered recommendations of colleagues. 

We abstracted the data as follows. One of us conducted the 
literature search on December 4, 2013 and determined which 
publications to include based on the abstract or full text. For 
included publications, the reviewer recorded the geographic 
setting, data timeframe, whether the focus was seasonal or 
pandemic influenza, details of the input data and analytical 
methods, and reported forecast accuracy. Another reviewer 
independently abstracted the data from selected publications. 
The two reviewers resolved discrepancies through consensus. 

Results 

We included 35 publications in the review [10-44] (Figure 1). 
Twenty six (74%) of the studies were published in 2009 or later, 
with more than one-third published in 2012 or 2013. The studies 
fell into 3 categories based on the epidemiological application: 
population-based seasonal influenza forecasting (N = 27 publica- 
tions), medical facility-based forecasting of patient counts for 
seasonal or pandemic influenza (N = 4), and regional or global 
spread forecasting for pandemic influenza (N = 4) (Table 1). Most 
studies included areas of North America (N= 15) or Europe 
(N= 14 publications), while a few included areas in the Asia- 
Pacific region (N = 4) or had global scope (N = 3) (Table 1). 

Twenty-eight studies employed temporal forecasting without a 
spatial component, while 7 made forecasts in time and space 
(Table 1). The studies used diverse forecasting methods, with 18 
using statistical approaches without models for epidemiological 
processes and 17 employing epidemiological models (Table 1). 

Among the statistical approaches, methods included time series 
models (N = 9 publications), generalized linear models (N = 4), 
Bayesian networks (N = 2), classification methods (N = 2), survival 
analysis (N= 1), and a prediction market (N= 1) (Table 1). The 
mechanistic approaches included compartmental models, which 
model transitions across various sub-populations (susceptible- 
infectious-removed [SIR] models and variants) (N= 12 publica- 
tions); and agent-based models (ABMs), which model exposure, 
infection, transmission and behaviors for each individual in the 
population (N = 5) (Table 1). 

Several studies coupled mechanistic models with methods to 
update parameter estimates and forecasts as new influenza-related 
surveillance data becomes available. Nsoesie et al. [11,13] 
developed a simulation-optimization algorithm for their ABM, 
which iteratively proposes estimates of key epidemiological 



parameters, uses those estimates to simulate the future course of 
the epidemic, and compares observed surveillance data to forecasts 
to revise the parameter estimates. Ong et al. [25] and Shaman et 
al. [10,16] used data assimilation techniques to incorporate 
influenza-related surveillance data into their compartmental 
models and update parameter estimates and forecasts. 

The studies used dynamic virological (N=14), syndromic 
influenza-like illness (ELI; N = 1 3) and other influenza-related 
surveillance data to forecast influenza activity (Table 2). Birrell et 
al. [19] included serological data to model pre-existing immunity, 
as well as virological and syndromic data. Four studies included 
internet search query data (Google Flu Trends) [10,1 1,16,37]. Six 
studies considered meteorological data [10,16,26,37-39], with 3 
including the meteorological predictors in the final forecasting 
model [10,16,26]. 

While most studies reported various modeling outcomes, such as 
ILI time series, the specific outcomes used in model validation 
varied. Among the 27 population-based forecasting studies, 16 
used weekly predictions of weekly incidence 1 or more weeks into 
the future in the validation (Table 3). Nine studies predicted the 
timing of the epidemic peak or incidence at the peak; all 
performed validation using at least some forecasts made at least 
4 weeks before the actual peak [10-13,16-18,29,31]. The facility- 
based forecasting studies used 1-step-ahead [37-39] or n-step- 
ahead [40] predictions of visit counts over step sizes of 1 day [40] 
to 1 month [39]. The regional or global pandemic spread 
forecasting studies used early data from the 2009 influenza 
A(HlNl)pdm09 pandemic to predict outcomes at national level 
across countries, including pandemic arrival, and peak incidence 
and time of peak. 

The studies used various metrics for validation of forecasts 
against independent data, with mean (or median) absolute error 
and mean absolute percent error the most common metrics for 
forecasts of incidence (i.e., daily, weekly, or monthly incidence; 
peak incidence; or cumulative incidence) (Table 4; studies 
forecasting peak week or epidemic duration reported the time 
difference between predicted and observed values in the valida- 
tion). Among all studies, only 2 reported accuracy as a function of 
estimated forecast variance [10,16]. 

Comparing the accuracy of the forecasting applications is 
difficult because forecasting methods, forecast outcomes, and 
reported validation metrics varied widely. While many studies 
compared models with different sets of predictors, only 2 
compared distinct modeling approaches. Shaman et al. [10] 
compared their susceptible-infectious-recovered-susceptible (SIRS) 
compartmental model, coupled to an ensemble-adjusted Kalman 
filter (SIRS-EAKF), to various resampling approaches using 
previous influenza seasons. The SIRS-EAKF model was consid- 
erably more accurate in predicting ILI peak week for the 2012- 
2013 season across 108 US cities. Merler et al. [43] compared the 
performance of an ABM and a simpler compartmental model in 
predicting the course of the 2009 influenza A(HlNl)pdm09 
pandemic in Europe, and found the simpler model failed to predict 
pandemic dynamics and attack rate accurately across countries. 

Among the 17 studies that used epidemiological models, 8 
provided results of sensitivity analysis for clinical, epidemiological, 
demographic, and other parameters [10,12,16,19,34,41-43]; 
however, Nsoese et al. published a sensitivity analysis separately 
[45] for the ABM used in publications included in this review 
[11,13,22]. 
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2,846 publications identified through 
database searches 
(PubMed, CINAHL, Project Euclid, Cochrane 
Database of Systematic Reviews) 



6 publications identified through manual 
searches of personal bibliographies, 
colleague recommendations, and the 
International Journal of Forecasting 



2,511 publications retained for 
title/abstract review 



141 publications retained for 
full-text review 









35 publications retained for 
manual search of references 



T 



335 duplicate publications 
removed 



2,374 publications excluded 
because studies were non-human 
or predictions were for patient 
infection or prognosis, viral 
characteristics, or population 
knowledge, attitudes, or practices 



106 publications excluded 
because studies did not perform 

or show results of validation 
against independent data or were 
"nowcasting" and not forecasting 



6 additional publications 
identified and excluded 



35 publications included in 
scoping review 



Figure 1. Literature search flow. 

doi:10.1371/journal.pone.0094130.g001 



Discussion 

This review shows accelerating publication of influenza 
forecasting methods in recent years. We identified diverse 
modeling applications to forecast influenza and ILI activity in 
human populations, including various purely statistical approaches 
and methods based on mechanistic (i.e., epidemiological) model- 
ing. Most models predicted influenza activity in a specific 
population, while several others predicted presentations at medical 
facilities or regional or global pandemic spread. Several models 
incorporated additional data besides clinical or laboratory-based 
surveillance data to generate forecasts, including internet search 
queries and meteorological data. The outcomes predicted and 
metrics used in validation varied. Most studies using mechanistic 
models did not present a sensitivity analysis for key epidemiolog- 
ical assumptions. 

The review provides an overview and assessment of influenza 
forecasting, describing current approaches and highlighting 
research needs for this promising new domain of public health 
preparedness. Since the focus was on the use of models to forecast 
influenza activity, we included only studies that validated models 
against independent data, a crucial part of predictive model 
development since using the same data for model fitting and 
testing inflates estimates of predictive skill [46]. This approach 



complements the review of Nsoesie et al. [7] , which did not apply 
this restriction and provided a more detailed consideration of the 
outcomes predicted and advantages and disadvantages of the 
modeling methods employed. 

The study has some limitations. We cannot exclude the 
possibility we failed to identify relevant studies, though we used 
broad search terms and searched multiple databases, and we 
would not have identified newer studies described only in 
conference proceedings, or unpublished studies. Papers correctly 
(based on our criteria) excluded may yet prove useful for influenza 
forecasting, and further review of these may suggest new 
methodologies for generating influenza predictions. The review 
also cannot serve as a definitive guide to forecasting approaches 
with greater predictive skill, since settings and methodologies 
varied widely and only 2 studies [10,43] compared distinct 
modeling approaches. We approached the review as a scoping, 
rather than a systematic, review because of this diversity. Also, the 
purpose was not to offer detailed critiques of modeling method- 
ologies. Such an assessment would be useful, but we believe that a 
more broadly-based review of forecasting applications provides 
necessary context for this and other more focused assessments. 

The results suggest several areas of practice and research to 
advance influenza forecasting in human populations (Figure 2). 
First, developers of influenza forecasting models and technologies 
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Table 1. Overview of influenza forecasting studies. 





Ref. 


Influenza Application 


Setting 


Forecast Type 


Forecasting Method 


Population-based forecasting studies 


[10] 


Seasonal 


United States 


Temporal 


Mechanistic (compartmental model) 


[11] 


Seasonal 


Seattle 


Temporal 


Mechanistic (ABM) 


[12] 


Seasonal 


Montreal 


Temporal 


Mechanistic (ABM) 


[13] 


Unspecified 


Montgomery Co., VA; Seattle; Miami 


Temporal 


Mechanistic (ABM) 


[14] 


Pandemic (2009) 


New Zealand 


Temporal 


Mechanistic (compartmental model) 


[15] 


Seasonal 


Germany 


Spatial-temporal 


Statistical (time series model} 


[16] 


Seasonal 


New York City 


Temporal 


Mechanistic (compartmental model) 


[17] 


Seasonal 


Slovenia 


Temporal 


Statistical (GLM, regression tree) 


[18] 


Pandemic (2009) 


Italy 


Temporal 


Mechanistic (ABM) 


[19] 


Pandemic (2009) 


London 


Temporal 


Mechanistic (compartmental model) 


[20] 


Seasonal 


United States 


Temporal 


Statistical (GLM) 


[21] 


Pandemic (2009) 


Japan 


Temporal 


Mechanistic (compartmental model) 


[22] 


Unspecified 


Los Angeles; New York; Seattle 


Temporal 


Statistical (classification) 


[23] 


Pandemic (2009) 


Japan 


Temporal 


Mechanistic (compartmental model) 


[24] 


Seasonal 


Germany 


Spatial-temporal 


Statistical (time series model) 


[25] 


Pandemic (2009) 


Singapore 


Temporal 


Mechanistic (compartmental model) 


[26] 


Seasonal 


Hong Kong; Maricopa Co., AZ 


Temporal 


Statistical (time series model) 


[27] 


Seasonal 


2 US jurisdictions (not identified) 


Temporal 


Statistical (Bayesian network} 


[28] 


Seasonal 


United Kingdom (boarding school) 


Temporal 


Mechanistic (compartmental model) 


[29] 


Seasonal 


Sweden 


Temporal 


Statistical (GLM) 


[30] 


Seasonal 


United Kingdom 


Temporal 


Statistical (time series model) 


[31] 


Pandemic (1918, 1957, 1968) 


United Kingdom 


Temporal 


Mechanistic (compartmental model) 


[32] 


Seasonal 


Iowa 


Temporal 


Statistical (prediction market) 


[33] 


Seasonal 


Massachusetts 


Temporal 


Statistical (Bayesian network) 


[34] 


Seasonal 


United States, United Kingdom 


Temporal 


Mechanistic (compartmental model) 


[35] 


Seasonal 


France 


Spatial-temporal 


Statistical (time series model} 


[36] 


Seasonal 


Scotland 


Temporal 


Statistical (GLM) 


Facility-based forecasting studies 


[37] 


Seasonal 


Baltimore 


Temporal 


Statistical (time series model) 


[38] 


Pandemic (2009) 


Washington, DC 


Temporal 


Statistical (time series model} 


[39] 


Seasonal 


Baltimore 


Temporal 


Statistical (time series model) 


[40] 


Seasonal 


Barcelona 


Temporal 


Statistical (time series model) 


Regional 


or global pandemic spread forecasting studies 






[41] 


Pandemic (2009) 


Global 


Spatial-temporal 


Mechanistic (compartmental model) 


[42] 


Pandemic (2009) 


Global 


Spatial-temporal 


Mechanistic (compartmental model) 


[43] 


Pandemic (2009) 


Europe 


Spatial-temporal 


Mechanistic (ABM) 


[44] 


Pandemic (2009) 


Global 


Spatial-temporal 


Statistical (survival analysis) 



GLM, generalized linear model; ABM, agent-based model. 
doi:1 0.1 371 /journal.pone.00941 30.t001 



should adhere to good practices in development, implementation, 
application, and description of epidemiological models. One 
possible guide [47], developed for veterinary epidemiology but 
applicable to studies in human populations, provides several 
recommendations that could facilitate comparison and implemen- 
tation of influenza forecasting models and technologies. These 
include use of sensitivity analysis to assess dependence of the model 
to all chosen parameter values and assumptions, and provision of 
the computer code implementing the model (in the publication or 
on request). 



However, in light of the methods identified in our review, some 
modification to these and related guidelines may be appropriate 
for influenza forecasting. For example, some approaches (e.g., 
[10,11,13,16,25]) optimize parameter values iteratively, as part of 
the forecasting algorithm; model developers do not explicidy 
assign parameter values or distributions. We encourage developers 
and users of epidemiological forecasting models to develop 
common, recommended practices for the field. 

Second, there is a need for comparisons of diverse forecasting 
models using common input data and validation approaches and 
metrics. While some of the general advantages and disadvantages 
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Table 2. Dynamic surveillance data used in forecasting studies. 





Ref. 


Data Timeframe 


Influenza Data 






Meteorological Data 






Virology 


ILI 


Other 




Population-based forecasting studies 


[10] 


2012-3 




* 


Google Flu Trends 


* 


[11] 


2007-8, 2012-3 






Google Flu Trends 




[12] 


2001-6 










[13] 


NA (simulated data) 






Simulated incidence 




[14] 


2009 


* 








[15] 


2001-8 


* 








[16] 


2003-5, 2007-9 






Google Flu Trends 


* 


[17] 


2006-2009 




* 


Medication sales 




[18] 


2009 




* 






[19] 


2009-10 


* 


* 


Serology 




[20] 


1 997-2009 


* 


* 






[21] 


2009-10 




# 






[22] 


NA (simulated data) 






Simulated incidence 




[23] 


2009-10 






Medication prescriptions 




[24] 


2001-8 


* 








[25] 


2009-10 




# 






[26] 


2004-9 


* 






# 


[27] 


2003 










[28] 


1978 






Confined to bed 




[29] 


1998-2006 


* 








[30] 


1992-2005 


* 








[31] 


1918-9, 57-8, 68-70 






Influenza deaths 




[32] 


2004-5 






Prediction market trades 




[33] 


1998-2000 










[34] 


2001-2 (US), 2003-4 (UK) 










[35] 


1 984-2002 




* 






[36] 


1972-99 




* 






Facility-based forecasting studies 


[37] 


2004-1 1 


* 




Google Flu Trends 


* 


[38] 


2009-1 1 


* 






* 


[39] 


2002-2008 


* 






* 


[40] 


2004-2008 




* 






Regional or global pandemic spread forecasting studies 


[41] 


2009 






Pandemic emergence 




[42] 


2009-10 






Pandemic emergence 




[43] 


2009 






Pandemic emergence 




[44] 


2009 






Pandemic emergence 





ILI, influenza-like illness. 

doi:1 0.1 371 /journal.pone.00941 30.t002 



of various ILI forecasting approaches have been identified [7], 
direct comparisons would yield insight into methods that perform 
better than others under particular circumstances. Such initiatives 
are underway at the Centers for Disease Control and Prevention 
[48], Intelligence Advanced Research Projects Activity, and 
Department of Defense, and could help guide future efforts. 
Head-to-head comparisons of automated detection algorithms to 
identify disease outbreaks in syndromic surveillance data [49] also 
could be a useful example for comparing forecasting methods. 



Third, assessments of forecasting methods should demonstrate 
how the accuracy of the method varies and should quantify this 
variability for use in real-time prediction. That is, it is not sufficient 
merely to predict an event; the likelihood of that prediction should 
also be ascribed. This quantification of likelihood, or expected 
accuracy, mirrors practices used in numerical weather prediction- 
e.g., a forecast of an 80% chance of rain tomorrow is a highly 
calibrated prediction of the likelihood of an event. We believe this 
aspect of model performance - calibration - will be a key 
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Table 3. Forecast outcomes used in model validation. 





Outcome 


Number of studies (refs.) 


Population-based forecasting studies 


Weekly incidence 


1 6 [1 2,1 3,1 5,1 7-1 9,21 ,23-26,30,32-35] 


Daily incidence 


3 [14,27,28] 


Peak time and/or incidence 


9 [10-13,16-18,29,31] 


Cumulative incidence 


3 [13,20,36] 


Epidemic duration 


2 [12,31] 


Facility-based forecasting studies 


Monthly visits 


1 [39] 


Weekly visits 


1 [40] 


Visits over 3 days 


1 [38] 


Peak visits 


1 [37] 


Regional or global pandemic spread forecasting studies 


Peak incidence (national) 


1 [43] 


Time of pandemic arrival (national) 


1 [44] 


Time of peak (national) 


2 [42,43] 


Cumulative incidence (U.S.) 


1 [41] 



doi:1 0.1 371 /journal.pone.00941 30.t003 



consideration for practitioners who might use a forecasting model 
in an operational setting. Reporting the range or confidence 
intervals associated with predicted outcomes is essential in 
validation studies, but this alone does not help a user determine 
how much certainty a specific forecast warrants. 

Fourth, future operational forecasting efforts should develop 
explicit approaches that incorporate additional expertise and 
analysis from scientists and public health officials. (The only 
documented systematic elicitation of expert judgment, for any type 
of modeling approach, in our review was the prediction market of 
Polgreen et al. [32].) Similar methods exist in weather and climate 
forecast (e.g., [50]). For example, meteorological forecasts are 
typically statistically post-processed to account for inherent model 
biases, and new methods for this post-processing are still being 
developed [51]. These combined results are then further vetted by 
meteorologists to monitor anomalous prediction behavior, and 
communicated to the public and decision makers. Infectious 
disease forecasting will need to explore and develop analogous 



frameworks for the post-processing of multiple forecast streams, 
the monitoring and calibration of these probabilistic forecasts, and 
the communication of these predictions to public health officials 
for decision support. 

Fifth, now that diverse ILI forecasting approaches are available 
and some have demonstrated promising performance in validation 
studies, assessments of real-world applications could spur the 
transition of these approaches to public health practice. Pilot 
studies in health departments, medical facilities, or other settings 
could assess forecasting applications not only for predictive skill, 
but for user acceptance, contributions to public health decision- 
making, and other outcomes at the user-model interface. 
Evaluations should compare various modeling approaches on 
these key characteristics, to identify approaches useful (not just 
accurate) in real-world settings. For example, forecasting ILI time 
series or peak week could be useful for anticipating needed surge 
capacity of personnel and materials, but modeling methods that 



Table 4. Validation metrics used in incidence forecasts. 





Metric 


Number of studies (refs.) 


MAE or MdAE 


6 [21,25,29,30,39,41] 


MAPE 


5 [12,27,31,32,36] 


RMSE 


5 [13,26,35,38,39] 


Correlation or t-test 


5 [13,20,27,33,35] 


95% CI 


4 [11,13,38,43] 


Scoring rules 


2 [15,24] 


Forecast confidence 3 


1[37] 


No quantitative metric 


8 [14,17-19,23,28,34,40] 



MAE, Mean absolute error; MdAE, Median absolute error; MAPE, Mean absolute percent error; RMSE, Root mean square error. 

a Forecast confidence was defined as "the percentage of forecast values within a predefined difference of the actual data during an influenza peak (here chosen as 20% 
of the mean of the maximal point of the influenza peak)." 
doi:1 0.1 371 /journal.pone.00941 30.t004 
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• Development of best practices for forecasting studies. 

• Head-to-head comparison of varied forecasting methods 
using common datasets and validation approaches. 

• Assessment of model calibration. 

• Methods to incorporate subjective input into 
quantitative forecasting models. 

• Pilot studies to assess usefulness in real-world settings. 

• Improved mutual understanding and routine 
collaboration among modelers and public health officials. 

Figure 2. Some needs for advancement of influenza forecast- 
ing. 

doi:10.1371/journal.pone.0094130.g002 

permit re-estimation of outcomes under various response scenarios 
could provide additional support to decision-makers. 

Last, model developers and decision-makers must understand 
each other's work better. Developers are more likely to provide 
useful tools if they know the key decisions users will make in 
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preparing for or responding to influenza outbreaks. They can 
develop and evaluate models around those specific decisions. To 
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become familiar with the modeling tools they might use, and 
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Efforts to link modelers and public health officials through 
seminars, on-the-job observation, exercises, and other activities 
could foster this mutual understanding and improve collaboration 
during emergencies. 
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